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Description 

Background of the Invention 
Field of the Invention 

[0001] The invention relates to a representative im- 
age display apparatus comprising a series of multiple 
images into which a motion image is formed, and which 
displays a representative image of each scene, and to 
a motion image search apparatus which executes 
searches for a motion image using the representative 
image display apparatus. 

Description of Prior Art 

[0002] Generally, a motion image contains multiple 
scenes. Each scene has independent meaning, and is 
defined by a group of consecutive images. Fig. 2 shows 
a situation where three scenes have been recorded on 
a video tape. Scene A is a view of a person, scene B is 
of a house, and scene C is of an automobile. Each scene 
respectively comprises two (A1 and A2), two (B1 and 
B2), and three (C1 -C3) images. A motion image search 
apparatus is used to quickly select a required scene 
from a motion image on recorded media, such as video 
tapes, when editing motion image images such as those 
required in a multimedia presentation creation or in 
broadcasting. Recently, apparatus have been devel- 
oped that display a representative image (hereinafter re- 
ferred to as representative image) of each scene after 
a motion image has been partitioned into scene units, 
and that allow a desired scene to be selected using the 
representative image as an index. 
Fig. 3 is a block diagram from a catalog for "Mediachef/ 
CUT" by Hitachi, Ltd. showing a system using the prod- 
uct. Details of its operation are given in "Mediachef/CUT 
for Windows User Operation Manual" (Windows is a reg- 
istered trademark). 

In the figure, the system comprises a PC 2 which per- 
forms general control, a video playback apparatus 4 
which provides the motion image to the PC 2, and a color 
printer 6 which prints the edited images. The PC 2 con- 
tains a video capture board 8 which captures and digi- 
tizes images provided from a video playback apparatus 
4. This configuration performs, for example, the follow- 
ing steps. 

(1 ) First the motion image to be processed is loaded, (2) 
a transition point in the scene is automatically identified 
based on the degree of color change in the images, (3) 
an image is decided to be a representative image that 
appears immediately after the transition point is ob- 
served, and (4) the representative image is stored or 
printed as an index for each scene. In the case of Fig. 
2, a scene transition point is identified between A2 and 
B1 , and between B2 and C1 , and the leading images of 
each scene, B1 and C1 , become the representative im- 
ages. 



[0003] This system helps the user in tape editing, for 
example, by printing out the representative images con- 
tained in one video tape cassette in a small sheet and 
affixing the sheet to the side of the video tape cassette. 
5 In addition, editing processes such as sorting, deletion, 
and trimming of the scenes are possible on the PC 2. 
[0004] In the system described above, referencing the 
representative images allows the user to lookfordesired 
scenes and replay them, that is, to search for a motion 
10 image. However, in representative images where a ship 
is afloat on the ocean, for example, the direction of the 
ship's movement can only be determined by replaying 
the motion image. If several representative images of 
this sort exist, for example, when searching for pictures 
15 where the ship moves to the right, the user must check 
each scene containing the representative image one by 
one. This manual search is not only laborious when 
there are a large number of similar representative imag- 
es, but inconvenient when one tries to grasp the general 
20 movement of a motion image, and leaves much room 
for improvement for efficient editing. 
[0005] Proc. from Interchi' 93, Conference on Human 
Factors in Computing Systems, 24 April 1993, page 
525, Article by Ueda et al: "IMPACT: Interactive Motion- 
25 Picture Authoring System for Creative Talent" already 
discloses a representative image display apparatus 
which comprises: an image input section for inputting 
and storing a motion image comprising a sequence of 
multiple images; a scene partition section for analyzing 
30 the input motion image, detecting transition points in the 
scenes, and partitioning the motion image into scenes; 
a representative image selection section for selecting a 
proper image from the motion image and setting it as a 
representative image; a motion decription image crea- 
35 tion section which detects the characteristics of the 
movement of the filmed object or the camera work 
(zooming and panning); a change information hold sec- 
tion for associating and holding the information related 
to the motion description image for each schene; and a 
40 display control section for displaying the representative 
image of the scene with its associated change informa- 
tion. Specifically, a representative image is selected for 
each cut of the motion picture and arranged in a time 
sequence order, and classification of each cut is repre- 
ss sented by an arrow as the motion description image. 
More specifically, a motion is classified into patterns, 
and the directions, colors and the like of the arrows are 
changed based on the classification. 



[0006] It is the object of the present invention to pro- 
vide a representative image display apparatus and a 
motion image search apparatus, that enable the user to 
55 easily and intuitively grasp the contents of a motion im- 
age. 

[0007] This object according to the invention is solved 
for the representative image display apparatus by the 
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features of claim 1 and for the motion image search ap- 
paratus by the features of claim 4. Preferred embodi- 
ments of the representative image display apparatus 
are defined in claims 2 and 3. 

[0008] The representative image display apparatus of 5 
the invention includes an image input section which in- 
puts and stores a motion image comprising a sequence 
of multiple images, a scene partition section which an- 
alyzes the input motion image, detects scene transition 
points, and partitions the motion image into scenes, a 10 
representative image selection section which selects a 
proper image as the representative image from among 
the images comprising each scene, a motion description 
image creation section which creates an image visually 
showing the movement of an object based on a differ- 15 
ence among a plurality of images, a change information 
hold section which associates and holds the information 
related to the motion description image for each scene 
with the representative image of the scene, and a dis- 
play control section which combines and displays the 20 
representative image of at least one scene with its as- 
sociated change information. 

[0009] In this configuration, the motion image which 
was input into the apparatus is first stored by the image 
input section. Next, the motion image is read, its scene 25 
transition points detected, and it is partitioned into 
scenes. Then, both a representative image of each 
scene is created and the information on changes occur- 
ring in an object in every scene is associated and held 
with the representative image of the corresponding 30 
scene. Finally, a representative image of at least one 
scene and the change information that was associated 
with the representative image are combined and dis- 
played. Thus, the movement of an object can easily be 
grasped. 35 
[0010] Naturally, the "movement of the object" may 
occur as a result of movement at the camera. A "differ- 
ence" between images is determined, for example, by 
obtaining the difference in pixel values at corresponding 
locations. 40 
[0011] Further, the above motion description image 
creation section fetches, at a preset time interval, the 
image stored in the image input section and creates its 
outline image. The outline image is then superimposed 
onto a previously created motion description image so 45 
as to create a new motion description image, and the 
fetch, the creation of the outline image and the super- 
imposition processes are repeated to create new motion 
description images in succession. The "preset time in- 
terval" can be specified from an external source or can 50 
be the default value of the apparatus, in this aspect, the 
outline images at the various times are all shown in one 
image. Only the outline image that was created first may 
be used without change as a motion description image. 
[0012] Accordingly it is even easier to grasp the move- 55 
ment. 

[0013] Further, when creating outline images at pre- 
set time intervals, the above motion description image 
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creation section creates new motion description images 
while changing the colors of the outlines based on the 
elapsed time from a scene's start time. Here "color" in- 
cludes black, white, and neutral tints (or achromatic 
shades). 

[0014] Thus, not only the movement but also the di- 
rection of movement can be easily grasped. 
[0015] In another embodiment of the invention, when 
an arbitrary time between the scene start time and end 
time is specified, the above display control section high- 
lights the location of the object at the specified time in 
the above motion description image. Since the "location 
of the subject" is indicated from the outline image, one 
application of the aspect is to change the intensity of 
outline image that corresponds to the specified time. 
[0016] According to this embodiment, it becomes 
easy to grasp the location of the object at a certain time, 
and this can facilitate the search for the search start po- 
sition. 

[0017] The motion image search apparatus of the in- 
vention performs searches for motion image and in- 
cludes one of the above representative display appara- 
tus, a video source which plays back and supplies a mo- 
tion image to the above representative display appara- 
tus, a video source control section which controls the 
playback of the motion image by the video source, and 
a command input section which inputs commands for 
search processing. In this configuration, when one 
scene and a playback start time from the displayed 
scenes are specified by the command, the video source 
control section follows the command and performs play- 
back of the motion image from the above playback start 
time of the specified scene. The playback enables the 
user to confirm the search result. According to the mo- 
tion image search apparatus of the invention, all the ad- 
vantages of the above representative image display ap- 
paratus are obtainable and the motion image editing ef- 
ficiency is greatly improved. 

Brief Description of the Drawings 

[0018] 

Fig. 1 is an overall block diagram of the motion im- 
age search apparatus concerning embodiment 1. 
Fig. 2 exemplifies a state where three scenes have 
been recorded on video tape. 
Fig. 3 is a block diagram, taken from a catalog for 
Mediachef/CUT, of a system using "Mediachef/ 
CUT". 

Fig. 4 is a block diagram of motion description im- 
age creation section 123 of embodiment 1. 
Fig. 5 is a flowchart showing the flow of processes 
executed by the motion description creation section 
123 of embodiment 1 . 

Fig. 6 shows a Ul which is displayed at display sec- 
tion 1 04 of embodiment 1 . 

Fig. 7 is a flowchart showing the flow of processes 
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which use the Ul of Fig. 6. 

Fig. 8 is an enlargement of the motion description 
image display area included in Fig. 6. 
Fig. 9 is a block diagram of a motion description im- 
age creation section of embodiment 2. 
Fig. 1 0 is a flowchart showing the flow of processes 
executed by the motion description image creation 
section 123 of embodiment 2. 
Fig. 11 shows a result obtained when the motion 
description image of Fig. 8 is recreated by the ap- 
paratus of embodiment 2. 

Fig. 1 2 is a block diagram of a catalog creation sec- 
tion 120 of embodiment 3. 

Fig. 1 3 is a block diagram of a catalog storage sec- 
tion 130 of embodiment 3. 

Fig. 14 shows the Ul which is displayed at display 
section 104 of embodiment 3. 

Description of Preferred Embodiments 

[001 9] Here, preferred embodiments of a representa- 
tive image display apparatus and a motion image search 
apparatus incorporating the representative image dis- 
play apparatus are described. 

This description will also clarify the representative im- 
age display method of the invention. In the motion image 
search apparatus described in the embodiments below, 
a playback apparatus which plays back a video medium 
is included, and the playback apparatus is electrically 
controlled directly from a main control section in the mo- 
tion image search apparatus. Although this configura- 
tion is ideal, for example, in broadcasting where rela- 
tively advanced functions are required, its applications 
are of course not limited thereto. 

Embodiment 1 

[0020] In this embodiment, an apparatus is described 
which creates an image showing the motion of an object 
in every scene and displays it together with a represent- 
ative image. 

Configuration 

[0021 ] Fig. 1 is an overall block diagram of the motion 
image search apparatus concerning this embodiment. 
Since a main control section 1 1 0 controls the overall ap- 
paratus in this figure, this and a command input section 
103 mentioned later correspond to the representative 
image display apparatus. A video source 1 01 plays back 
recorded video media such as those for VCRs and pro- 
vides motion image to the main control section 110 and 
a motion image display section 102. The main control 
section 110 creates representative images and the mo- 
tion description images from the provided motion image, 
and motion image display section 102 simply displays 
the provided motion image. The motion image display 
section 102, for example, a monitor which displays or- 



dinary NTSC format signals, allows a user to directly 
view the searched motion image. 
A video source control section 1 1 2 controls the playback 
of motion image from a video source 101. It performs 

5 control functions required for the search process, such 
as "stop after 10 seconds from the start", in addition to 
the various control functions (tape rewind, fast forward, 
pause, and so forth) available on the front panel of home 
VCRs. Control signals are connected to the video 

10 source 101 by a cable. A video source control section 
112 controls the playback of the motion image that is 
necessary for the main control section 110 to create a 
representative image, and necessary for the motion dis- 
play section 102 to display. 

15 The played back motion image is input by an image input 
section 111. Image input section 111 includes an A/D 
converter and aframe memory (not shown). Image input 
section 111 digitizes the input image, stores it into the 
frame memory and waits for further processing. The im- 

20 age input section 111 may also contain a disk storage 
apparatus which secondarily stores the contents of the 
frame memory. 

In this case, it becomes unnecessary to read the motion 
image required for processing every time from video 

25 source 1 01 , and processing efficiency can be improved. 
A catalog creation section 120 is composed of a series 
of software modules which read image data from the 
frame memory, and creates a catalog based on this da- 
ta, and which comprises a scene partition section 121, 

30 a representative image selection section 1 22, and a mo- 
tion description image creation section 123. A catalog 
is a list of information concerning multiple scenes. On 
the other hand, based on the contents of the data output 
from the catalog creation section 120, a catalog storage 

35 section 130 sorts and stores the data, and comprises a 
scene partition data storage section 131 , a representa- 
tive image storage section 1 32, and a motion description 
image storage section 133. 

The scene partition section 121 analyzes the digitized 
40 image using known methods and partitions the motion 
image into scenes based on the difference between two 
consecutive scenes. Based on the position (for example 
a time code or counter value such as LTC or VITC re- 
corded on a tape) on the medium at the partition point, 
45 the start point and end point of each scene are stored 
in a scene partition data storage section 131 . 
The representative image selection section 122 selects 
a certain frame from the scene as a representative im- 
age. The representative image may be a frame at the 
50 beginning, middle, or end of the scene, or a frame spec- 
ified by the user. To identify the scene which the repre- 
sentative image represents, information such a scene 
number is stored into the representative image storage 
section 132. 

55 Motion description image creation section 1 23 performs 
adistinctive process in this embodiment. Fig. 4 is a block 
diagram of a motion description image creation section 
123. 
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In this figure, an outline image creation section 201 proc- 
esses the input image and creates an outline image. A 
composite image creation section 202 performs com- 
posite processing on the created outline image and an 
outline image storage section 203 temporarily stores the 
composed outline image. In this embodiment, the out- 
line of an object included in a given image within a scene 
is sampled in each image, and these outlines are super- 
imposed to create one image. 

Command input section 103 permits the entry of com- 
mand inputs by the user and employs a device such as 
a keyboard, mouse, or various pointing devices. The in- 
put command is processed by an input management 
section 113, which is a command dispatcher, to initiate 
the required action within the main control section 110. 
A display control section 1 1 4 includes a system program 
for the display system which centrally controls the dis- 
play of various images such as representative images 
and motion description images, and a display circuit 
such as a VGA controller and its driver. In the display 
control section 1 1 4, the display state such as for the lo- 
cation of a catalog which is read from catalog storage 
section 130 is ultimately determined and displayed at a 
display section 1 04. The display section 1 04 may be an 
ordinary display apparatus on which representative im- 
ages and motion description images are displayed. 

Process Operations 

[0022] The main control section 110, which receives 
a scene partition start instruction based on command 
inputs, issues a play-back instruction to the video source 
101 through the video source control section 112. The 
motion image output from video source 1 01 is displayed 
by the motion image display section 102 and digitized 
in frame units by the image input section 111. 
Next, each frame is sent to a catalog creation section 
120. At the scene partition section 121, a histogram is 
taken for the input image and its preceding image, and 
a scene's partition point is judged when the absolute val- 
ue of its difference reaches or exceeds a threshold. This 
method itself is known. Instead of this method, any char- 
acteristic physical parameter between consecutive im- 
ages may be compared to detect the partition point. 
Once the partition point is ascertained, information such 
as a time code recorded on the video tape is acquired 
from the video source 101 and stored in the scene par- 
tition data storage section 131. In parallel with this, a 
representative image is created and stored as a still im- 
age file in the representative image storage section 1 32. 
Next, the motion description image is created. 
Fig. 5 is a flowchart showing the flow of processes by 
motion description image creation section 123. Accord- 
ing to this figure, the sent image is passed through a 
Compass-gradient filter in the outline image creation 
section 201 and an outline image is created (S211). In 
the outline image, each pixel takes on one of two values 
0 and 1 , where pixels on the outline become 1 and other 



pixels become 0. Outline images may be obtained at a 
user specified frame interval, for example, every 10 
frames, or at an interval preset as a default value in the 
apparatus. 

5 Next, the image which has been stored in the outline 
image storage section 203 is read and composed 

(5212) by the composite image creation section 202 
with the outline image created in S211. Composing of 
the image is achieved by taking the logical OR of pixel 

10 values at the same coordinates for all pixels in the two 
images. The composite image thus created is stored 

(5213) in the outline image storage section 203. These 
processes are repeated until a scene partition point is 
detected. When the scene partition point is detected, the 

15 image stored in the outline image storage section 203 
is transferred to and stored in the motion description im- 
age storage section 133. 

Here, the above processes are described together with 
user input operations and actual image display exam- 
20 pies. 

Fig. 6 shows a user interface (Ul) displayed by the dis- 
play section 104, and Fig. 7 is a flowchart showing the 
flow of processes using the Ul. In Fig. 6, a scene parti- 
tion start button 152 starts scene partition. A playback 

25 button 1 53 initiates playback of the motion image for the 
selected scene and a stop button 154 stops the motion 
image being played back. In Fig. 6, the entire screen 
above these buttons is a catalog display area 151 which 
is organized into 3 scene display areas 155, each cor- 

30 responding to one scene. Each scene display area 1 55 
contains a representative image display area 1 56 which 
displays the representative image of that scene, a mo- 
tion description image display area 157 which displays 
the motion description image, and a scene partition data 

35 display area 1 58 which displays each scene's start point 
(start time) and end point (end time). The first two digits 
of the time display indicate the hours, followed by min- 
utes, seconds, and frame number. The frame number 
for the NTSC format is from 0 to 29 since the number of 

40 frames displayed is 30 per second. When playing back 
a motion image, the user is allowed to specify before- 
hand how much to play back (that is, for how much of 
the motion image scene partition is to be performed). 
This specified amount is converted to a time code on 

45 the medium and held in the apparatus as the specified 
end position. 

In the Ul, once the user presses the scene partition start 
button 152, the input management section 113 inter- 
prets this command (S170) and a playback instruction 

50 (S1 71 ) is sent to the video source 1 01 . The medium in 
the video source 1 01 is played back (S1 72) and the mo- 
tion image is sent (S1 73) to the catalog creation section 
120. At this time, it is judged whether the time code on 
the medium indicates the specified end position (S1 74) 

55 or the status indicates the end position (S175) of the 
medium, and if either is realized the process ends 
(S176). 

If the process does not end (No at S1 75), an outline im- 
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age for that frame is created and a motion description 
image is created (S177). In parallel with this, the input 
image is judged (S1 78) as to whether it should be a rep- 
resentative image, and a representative image is creat- 
ed (S1 79), for example, when a frame specified before- 
hand by the user is to be processed. 
Next, it is judged (S180) whether the scene partition 
point is reached, and if it is, the motion description image 
is transferred (S182) to the motion description image 
storage section 132, and catalog display information is 
created (S183) to display (S184) scene partition data, 
the representative image, and the motion description 
image. Then the process ends. If the scene partition 
point is not reached, the next frame is read and the same 
process is repeated. 

From the steps above, a scene's representative image 
and motion description image are displayed on a display 
apparatus. An enlarged diagram of the motion descrip- 
tion image display area included in Fig. 6 is shown in 
Fig. 8, where the movement of the object extending over 
multiple images is expressed consecutively in one im- 
age. As a result, the user can understand the scene in 
general from the representative image, and in detail 
from the motion description image. Based on these 
types of information, the user can actually view the 
scene by selecting it on the screen, by using a mouse, 
for example. As shown in Fig. 6, the selected scene has 
its area frame highlighted. The above buttons become 
effective for the selected scene. That is, the playback 
button 1 53 enables playback of the scene and the stop 
button 1 54 enables playback to be stopped. In this man- 
ner, a highly efficient motion image search is achieved. 
The above is an overview of the embodiment. Possible 
improvements and variations for this embodiment are 
discussed in the following. 



Embodiment 2 

[0023] Although the motion description image was 
created in embodiment 1 by superimposing successive 

5 outline images, the superimposition in embodiment 2 is 
performed while varying the color of the outline image. 
For this reason, if the outline image concerning a sub- 
sequently input frame is displayed using a darker color, 
for example, it becomes easier to grasp the movement 

10 of the display. 

Fig. 9 is a block diagram of the motion description image 
creation section 123 in this embodiment. The difference 
between this figure and Fig. 4 is that a hue setting sec- 
tion 301 is added. Hue setting section 301 varies the 

15 hue of the outline image with the lapse of time. 

Fig. 1 0 is a flowchart showing the flow of processes ex- 
ecuted by the motion description image creation section 
1 23. The difference between this figure and Fig. 5 is that 
S31 2 is inserted after S21 1 . That is, after an outline im- 

20 age is created (S21 1 ), the outline's hue is varied (S31 2) 
by the hue setting section 301 with the lapse of time from 
the start time of the scene. After this, the process is re- 
peated as shown in Fig. 5 and the motion description 
image is created. Gradation processing in which only 

25 the brightness is decreased can be employed in the 
processing of hue setting section 301 . In this case, the 
outline pixel values for the outline image obtained the 
first time are set to a maximum value of 255 on an 8-bit 
gray scale and decreased by 5 at a time, for example, 

30 in subsequently input images. Fig. 11 is an image ob- 
tained as a result of re-creating the motion description 
image of Fig. 8 using the apparatus of this embodiment, 
and it becomes easier to grasp the movement of the 
subject. 

35 

Embodiment 3 

[0024] In this embodiment, the input of an arbitrary 
time between the start time and end time of a scene 
highlights the outline image corresponding to the input 
time (hereinafter referred to as specified time) in the mo- 
tion description image for the scene. Furthermore, the 
medium is actually forwarded or reversed until the po- 
sition where the image corresponding to the specified 
time is found. 

Fig. 12 is a block diagram of a catalog creation section 
120 in this embodiment and Fig. 13 is a block diagram 
of a catalog storage section 130. A new block added in 
Fig. 12 is a time calculation section 401 which obtains 
the amount of change in time occurring when a time but- 
ton described later is pressed after the specified time 
was input for a scene. 

On the other hand, a new block in Fig. 1 3 is a time stor- 
age section 411 which stores, for every scene, the time 
calculated by the time calculation section 401 . The time 
storage section 411 is a buffer which can store at least 
one time value for every scene, and is useful when a 
search temporarily moves to another scene and then re- 



(1) Display the motion description information as 
characters, symbols, and so forth. 
Instead of using a motion description image, the di- 
rection of movement of the subject can be ex- 40 
pressed using characters such as "right" and "up", 
or using symbols such as "->" and "T". This can be 
achieved by judging the angle of the subject's 
movement vector. When expressing movement us- 
ing characters, symbols, and so forth, the process- 45 
ing time and memory capacity required for the im- 
age creation can be reduced. 

(2) Display the transition information other than 
movement 

When, for example, the object is a neon sign, the 50 
object itself does not move but its color changes. In 
this case, examining such differences as color or 
brightness in the object before and after the change 
allow information such as "becomes red", "be- 
comes white", or "becomes bright" to be displayed 55 
as text data beside the representative image. 
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turns back to the original scene, so that the user can 
determine which image had been referenced previously. 
Fig. 14 shows the Ul displayed by the display section 
1 04. The difference between this figure and Fig. 6 is that 
a display of "current time" is newly added. When a spec- 
ified time is input, the current time is initialized with this 
value. Subsequently, every time a time button is 
pressed, the current time increases or decreases. Time 
buttons comprise a previous time button 451 which 
specifies a previous time and a subsequent time button 
452 which specifies a subsequent time. 
Using the configuration above, the user first selects a 
scene. Here, the topmost scene is selected. Next, the 
specified time is entered, for example, through a key- 
board. In the figure, the allowable input is in the range 
of scene start time 00:00:00:00 to end time 00:02:05:20. 
For example, when 00:01 :02:28 is specified, the outline 
image with the closest time is highlighted. Images are 
input in 1/30 second units, and when an outline image 
is created for every image, for example, the time corre- 
sponding to the outline image (called an outline image 
time) takes on a discrete value in 1/15 second units. For 
this reason, the specified time and the outline image 
time usually do not match exactly, so the closest time is 
selected. The selected outline image time is stored in a 
time storage section 411 . 

In this embodiment, along with highlighting the outline 
image, the playback position of the actual medium is for- 
warded or rewound to the outline image time. Thus, the 
objectonthe medium appears at the position of the high- 
lighted outline image. As a result, the user can see 
where the object is located at the specified time, and if 
playback from that position is desired, this can be 
achieved simply by issuing a playback instruction. 
When the position at the specified time is different from 
where playback is to start, the above time buttons are 
pressed. The pressing of the button is judged by the in- 
put management section 1 1 3 and a new current time is 
calculated by the time calculation section 401 . Since the 
outline image time takes on a discrete value in 1/15 sec- 
ond units, the current time increases or decreases by 
1/15 second each time the button is pressed, and the 
outline image adjacent to the currently highlighted out- 
line image is highlighted. The playback position on the 
medium is also precisely adjusted. 
The contents of time storage section 411 are updated 
with the new current time. From the processes above, 
it becomes possible to set the start position of playback 
with extremely high precision. If the subsequent time 
button 452 is pressed at the final outline image time of 
the scene, or conversely, if the previous time button 451 
is pressed at the initial outline image time, these instruc- 
tions will be ignored. 

In this embodiment, it is necessary to consider that the 
motion description image is only one image. That is, 
once this image is created, it becomes necessary to add 
information to the motion description image regarding 
which outline corresponds to what time. For this reason, 



(1) motion description images having differing levels of 
brightness according to preset times are created using 
the method of embodiment 2, (2) the correspondence 
between the various levels of brightness and times is 

5 established, (3) the pixels having the brightness corre- 
sponding to the current time are scanned in the motion 
description image, and (4) all matching pixels are high- 
lighted. The result of these steps is that only the desired 
outline stands out. 

10 While there have been described what are at present 
considered to be preferred embodiments of the inven- 
tion, it will be understood that various modifications may 
be made thereto, and it is intended that the appended 
claims cover all such modifications. 

15 

Claims 

1 . A representative image display apparatus compris- 
20 ing: 

an image input section (111) for inputting and 
storing a motion image comprising a sequence 
of multiple images, 
25 a scene partition section (121) for analyzing the 

input motion image, detecting transition points 
in the scenes, and partitioning the motion im- 
age into scenes, 

a representative image selecting section (122) 
30 for selecting a proper image from the motion 

image comprising multiple scenes and setting 
it as a representative image, 
a motion description image creation section 
(123) which creates an image visually showing 
35 the movement of an object and fetches, at a 

preset time interval, the image stored in the im- 
age input section (111) and creates its outline 
image, and superimposes the outline image on- 
to a previously created motion description im- 
40 age to repeatedly create a new motion descrip- 

tion image, 

a change information hold section (130) for as- 
sociating and holding the information related to 
the motion description image for each scene 
45 with the representative image of said scene, 

and 

a display control section (114) for combining 
and displaying, for at least one scene, the rep- 
resentative image of the scene with its associ- 
50 ated change information, characterised in 

that 

said motion description image creation section 
(123) creates new motion description images while 
55 changing colors of the outlines based on the 
elapsed time from the start time of the scene. 

2. The representative image display apparatus ac- 
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schafft, und das Umrissbild uber ein vorher ge- 
schaffenes Bewegungsbeschreibungsbild 
uberlagert, um wiederholt ein neues Bewe- 
gungsbeschreibungsbild zu schaffen, 

5 einen Anderungsinformations-Halteabschnitt 

(130) zum Assoziieren und Halten der Informa- 
tionen, die auf das Bewegungsbeschreibungs- 
bild fur jede Szene mit dem reprasentativen 
Bild der besagten Szene bezogen sind, und 

10 einen Darstellungssteuerabschnitt (114) zum 

Kombinieren und Darstellen des reprasentati- 
ven Bildes der Szene mit seinen assoziierten 
Anderungsinformationen fur wenigstens eine 
Szene, 

15 

dadurch gekennzeichnet, dass 

der Bewegungsbeschreibungsbild-Schaffungsab- 
schnitt (123) neue Bewegungsbeschreibungsbilder 
schafft, wahrend Farben der Umrisse auf der 
20 Grundlage der von der Startzeit der Szene verstri- 
chenen Zeit geandert werden. 

2. Darstellungsvorrichtung fur representative Bilder 
nach Anspruch 1, worin der Darstellungssteuerab- 

25 schnitt (114) die Position eines Gegenstands in ei- 
nem Bewegungsbeschreibungsbild zu einer be- 
stimmten Zeit hervorhebt, wenn eine beliebige Zeit 
zwischen der Startzeit und der Endzeit einer Szene 
spezifiziert ist. 

30 

3. Darstellungsvorrichtung fur representative Bilder 
nach Anspruch 2, worin der Darstellungssteuerab- 
schnitt (114) nach Pixeln mit einer Farbe entspre- 
chend einer gegebenen Zeit abtastet und den Ort 

35 des Gegenstands in einem Bewegungsbeschrei- 
bungsbild hervorhebt durch Anderung der Farbe al- 
ler durch Abtastung gefundener Pixel. 



cording to claim 1 , wherein said display control sec- 
tion (114) highlights the position of an object in a 
motion description image at a specified time, when 
an arbitrary time is specified between the start time 
and end time of a scene. 

3. The representative image display apparatus ac- 
cording to claim 2, wherein said display control sec- 
tion (114) scans for pixels having a color corre- 
sponding to the given time, and highlights the loca- 
tion of the object in a motion description image by 
changing the color of all the pixels found by scan- 
ning. 

4. A motion image search apparatus for performing 
searches for a motion image, comprising: 

the representative image display apparatus ac- 
cording to one of claims 1 to 3, 
a video source (1 01 ) for playing back and sup- 
plying a motion image to said representative 
image display apparatus, 
a video source control section (1 1 2) for control- 
ling the playback of the motion image from the 
video source, and 

a command input section (103) for inputting 
commands for search processing, 

wherein the video source control section (112) per- 
forms, when one scene from displayed scenes and 
a playback start time for the scene are specified by 
a command, playback of the motion image from the 
playback start time of the scene based on the com- 
mand. 



Patentanspriiche 

1. Darstellungsvorrichtung fur representative Bilder, 
welche aufweist: 

einen Bildeingabeabschnitt (111) zur Aufnah- 
me und Speicherung eines bewegten Bildes 
mit einer Folge von mehreren Bildern, 
einen Szenenteilungsabschnitt (121 ) zum Ana- 
lysieren des eingegebenen bewegten Bildes, 
Erfassen von Ubergangspunkten in den Sze- 
nen und Teilen des bewegten Bildes in Szenen, 
einen Auswahlabschnitt (122) fur representati- 
ve Bilder zur Auswahl eines geeigneten Bildes 
aus dem bewegten Bild mit mehreren Szenen 
und zum Setzen von diesem als ein represen- 
tatives Bild, einen Bewegungsbeschreibungs- 
Bildschaffungsabschnitt (1 23), welcher ein Bild 
schafft, das visuell die Bewegung eines Gegen- 
stands zeigt, und in einem voreingestellten 
Zeitintervall das in dem Bildeingabeabschnitt 
(111) gespeicherte Bild holt und sein Umrissbild 



4. Bewegungsbild-Suchvorrichtung zur Durchfuhrung 
40 einer Suche nach einem Bewegungsbild, welche 
aufweist: 

die Darstellungsvorrichtung fur representative 
Bilder nach einem der Anspruche 1 bis 3, 
45 eine Videoquelle (101) zur Wiedergabe und 

Lieferung eines Bewegungsbildes zu der Dar- 
stellungsvorrichtung fur representative Bilder, 
einen Videoquellen-Steuerabschnitt (112) zur 
Steuerung der Wiedergabe des Bewegungsbil- 
50 des von der Videoquelle, und 

einen Befehlseingabeabschnitt (103) zur Ein- 
gabe von Befehlen fur die Suchverarbeitung, 

worin der Videoquellen-Steuerabschnitt (112), 
55 wenn eine Szene aus dargestellten Szenen und ei- 
ne Wiedergabe-Startzeit fur die Szene durch einen 
Befehl spezifiziert sind, die Wiedergabe des Bewe- 
gungsbildes von der Wiedergabe-Startzeit der Sze- 
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ne an auf der Grundlage des Befehls durchfuhrt. 
Revendications 

5 

1. Systeme d'affichage d'images representatives 
comprenant : 

une section d'entree d'images (111) pour entrer 
et stocker une image mobile comprenant une 10 
sequence d'images multiples, 
une section de division de scenes (121) pour 
analyser I'image mobile entree, detecter les 
points de transition dans les scenes, et diviser 
I'image mobile en scenes, 15 
une section de selection d'images representa- 
tives (1 22) pour selectionner une image correc- 
te a partir de I'image mobile comprenant des 
scenes multiples et la definir en tant qu'image 
representative, 20 
une section de creation d'images de descrip- 
tion du mouvement (123) qui cree une image 
representant, de maniere visuelle, le mouve- 
ment d'un objet et extrait, a un intervalle de 
temps predefini, I'image stockee dans la sec- 25 
tion d'entree d'images (111) et cree son image 
de contour, et superpose I'image de contoursur 
une image de description du mouvement creee 
precedemmentpourcreer, de maniere repetee, 
une nouvelle image de description du mouve- 30 
ment, 

une section conservant les informations de 
changement (130) pour associer et conserver 
les informations liees a I'image de description 
du mouvement pour chaque scene avec I'ima- 35 
ge representative de ladite scene, et une sec- 
tion de commande de I'affichage (114) pour 
combiner et afficher, pour au moins une scene, 
I'image representative de la scene et ses infor- 
mations de changement associees, caracteri- 40 
se en ce que 

ladite section de creation d'images de descrip- 
tion du mouvement (123) cree de nouvelles 
images de description du mouvement tout en 
modifiant les couleurs des contours sur la base 45 
du temps ecoule depuis le debut de la scene. 

2. Systeme d'affichage d'images representatives se- 
lon la revendication 1 , dans lequel ladite section de 
commande de I'affichage (114) met en evidence la 50 
position d'un objet d'une image de description du 
mouvement a un instant specifie, lorsqu'un instant 
arbitraire est specifie entre le debut et la fin d'une 
scene. 

55 

3. Systeme d'affiche d'images representatives selon 
la revendication 2, dans lequel ladite section de 
commande de I'affichage (114) recherche, a I'aide 



d'un balayage, les pixels presentant une couleur 
correspondant a I'instant donne, et met en evidence 
I'emplacement de I'objet d'une image de description 
du mouvement en modifiant la couleur de tous les 
pixels reperes lors du balayage. 

4. Systeme de recherche d'images mobiles pour ef- 
fectuer des recherches sur une image mobile, 
comprenant : 

le systeme d'affichage d'images representati- 
ves selon I'une quelconque des revendications 
1 a 3, 

une source video (101) pour relire et delivrer 
une image mobile audit systeme d'affichage 
d'images representatives, 
une section de commande de la source video 
(112) pour commander la reproduction de I'ima- 
ge mobile a partir de la source video, et 
une section d'entree d'instructions (103) pour 
entrer les instructions destinees a ce traitement 
de recherche, dans lequel 
la section de commande de la source video 
(112) realise, lorsqu'une scene parmi les sce- 
nes affichees et un temps de debut de lecture 
pour la scene sont specifies par une instruction, 
une lecture de I'image mobile a partir du temps 
de debut de la lecture de la scene sur la base 
de I'instruction. 
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